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Tactical  Naval  Applications  of  Advanced  3>D  Display  Technologies 

David  Rousseau 

Naval  Ocean  Systems  Center,  San  Diego,  CA 

Introduction 

A  group  of  rapidly  maturing  technologies  are  coming  together  to  permit  a  new 
way  of  displaying  and  controlling  information.  The  increasing  speed  and  decreasing 
cost  of  high-resolution  graphics  computers  is  making  it  practical  to  generate  complex 
imagery  with  acceptable  realism  in  real  time.  Small  high-resolution  monochrome 
cathode  ray  tubes  (CRTs)  have  long  been  used  for  military  helmet-mounted  displays. 
Liquid  cr/sta!  displays  (LCDs)  with  acceptable  resolution  are  just  now  becoming 
available.  Both  are  finding  their  way  into  commercial  stereoscopic  display  systems. 
Commercial  versions  of  position  sensors  developed  for  military  head-trackers  are 
becoming  increasingly  affordable,  and  their  use  has  been  expanded  to  create  three- 
dimensional  (3-D)  controllers.  Finally,  the  newest  technology  on  the  scene  is  digitally 
modified  sound  that  presents  the  operator  with  realistic  3-D  binaural  hearing.  These 
3-D  and  other  display  technologies  are  being  studied  for  their  application  to  tactical 
Naval  operations  at  the  Naval  Ocean  Systems  Center  (NOSC).  Among  the  potential 
beneficiaries  of  this  technology  are  airborne  early  warning  and  fonvard  air  control, 
platform  and  force  level  battle  management,  air  traffic  control,  compact  flight  trainers 
and  mission  planners,  and  many  aspects  of  anti-submarine  warfare  (ASW).  The 
mission  area  currently  being  investigated  at  NOSC  is  in  the  improvement  of  shipboard 
ASW  sensor  information  displays. 


Discussion 

It  is  known  that  performance  with  displays  improves  when  operator  workload  is 
reduced^  2. 3  and  that  this  effect  Is  amplified  by  prolonged  operations.^  This  is 
because  reduction  in  operator  workload  frees  up  cognitive  resources  which  can  then 
be  applied  to  other  tasks  such  as  improved  decision  making.^- 2  Therefore,  displays 
which  minimize  this  work  load  should  result  in  improved  operator  performance  and 
improved  decision  making.  The  need,  therefore,  is  for  displays  which  present 
information  in  ways  which  take  advantage  of  natural  human  perceptual  and  cognitive 
skills.  Few  display  systems  in  use  today  are  more  abstract  than  those  used  to  support 
the  ASW  mission  area. 

Sonar  displays  have  not  changed  significantly  for  roughly  ten  years.  Existing 
ASW  displays  present  information  that  is  frequently  truncated,  altered,  or  otherwise 
simplified  due  to  limitations  in  data  processing,  or  the  physical  limitations  of  the 
display  devices  themselves.  Currently,  diverse  sources  of  information  must  be 
correlated  by  the  sonar  operator  who  creates  a  mental  model  of  the  multi-dimensional 
ASW  environment.  The  majority  of  this  mental  image  must  be  derived  from  multi¬ 
dimensional  acoustic,  physical,  and  temporal  information  that  has  been  presented  in  a 
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very  abstract  visual  two-dimensional  (2-D)  format  from  a  number  of  sources  (Figure 
1).  This  method  of  presenting  complex,  cross- sensory,  multi-dimensional  information 
on  abstract  2-D  displays  may  soon  become  obsolete.  The  application  of  new  3-D 
visual  and  audio  display  technologies  to  this  problem  could  yield  significant 
improvements  in  operator  performance  and  operational  effectiveness.  These  3-D 
systems  could  present  the  critical  tactical  information  in  a  far  more  intuitive  and 
integrated  way,  thereby  reducing  the  cognitive  interpretation  burden  and  learning  time, 
while  improving  the  tactical  success  rate  over  current  systems.  Figure  2  depicts  the 
nature  of  the  relationship  between  cognition,  response  time,  and  error  rate.  When  the 
task  of  interpreting  information  is  complicated,  the  response  or  reaction  of  the 
individual  will  be  slower  and  the  error  rate  will  increase. 

An  advanced  technology  ASW  display  system  could  consist  of,  1 )  the 
application  of  3-D,  stereoscopic,  high-resolution,  helmet-mounted  or  boom-mounted 
displays,  2)  a  3-D  position  tracker  (for  the  HMD)  and  3-D  manipulators  for  computer 
function  control.  3)  the  incorporation  of  3-D  audio  for  presentation  of  multiple  beams 
from  the  sonar  for  Intuitive  cuing  and  for  correlation  of  acoustic  transients  and  active 
returns  to  the  ocean  environment,  and  4)  the  intuitive  depiction  of  high-resolution 
computer-generated  imagery  of  the  ASW  environment  based  on  integrated  display  of 
active,  passive,  and  environmental  sensor  information.  The  exact  nature  of  the 
displays  that  may  prove  most  effective  are  yet  to  be  determined,  but  speculation 
unfettered  by  display  constraints  leads  to  some  intriguing  possibilities. 

For  example,  in  the  active  sonar  mode  the  3-D  ASW  display  could  provide  the 
operator  with  a  360®  field-of-regard  image  of  the  ocean  bottom  topography  and  water 
properties  integrated  with  wave  fronts  representing  the  propagation  of  the  sonar 
pulses  (Rgure  3).  The  operator  would  see  these  wave  fronts  being  reflected  off  of  the 
local  topography  and  all  known  obstacles  in  that  dat?  base,  and  refracted  by  the 
model  of  the  local  water  properties.  The  operator  would  therefore  be  able  to  see 
which  returns  are  from  known  obstructions  and  compare  that  with  the  actual  returns 
received  by  the  sonar.  Integrating  this  3-0  visual  representation  with  acoustic  analysis 
and  cuing  from  the  3-D  audio  system  would  give  the  operator  an  intuitive  picture  of 
the  ASW  situation.  Similarly,  in  the  passive  sonar  mode  the  operator  would  be  able  to 
look  around  at  real-time  noise  sources  with  cuing  from  the  3-D  audio  system  (Figure 
4).  He  could  also  view  the  sonar  history  over  a  desired  time  span  for  all  bearings  and 
elevations  in  a  single  3-D  volume  (Figure  5).  In  each  case  any  sonar  beam  could  then 
be  designated  for  spectral  analysis. 

Similar  display  approaches  could  be  used  for  fixed  site  ASW  facilities. 
Application  of  such  a  display  system  to  airborne  ASW  is  more  challenging  due  to  the 
strict  size  and  weight  limits  of  ASW  aircraft,  but  computer  size  and  weight  are 
constantly  decreasing.  Many  of  the  displays  developed  for  surface  ASW  would  be 
directly  applicable  to  both  airborne  and  ashore  ASW.  A  few  specialized  displays  may 
be  added  for  such  things  as  sonobuoy  coverage  however  (Figure  6). 
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Significant  enhancements  to  total  system  effectiveness  could  result,  without 
redesign  of  other  system  components  (e.g.,  sensors  and  sources  of  self  noise). 

Recent  experiments  measured  significant  changes  in  sonar  operator  performance  due 
to  relatively  minor  modification  of  a  standard  sonar  display.®  These  experiments  were 
conducted  under  ideal  signal-to-noise  conditions  and  measured  a  multiple  db  loss 
due  to  a  proposed  fusion  of  data  on  a  conventional  ASW  display. 

As  this  research  continues,  new  methods  for  presenting  ASW  information  couic 
arise  that  would  make  data  relationships  more  apparent,  increase  situational 
awareness  and  enhance  the  quality  of  operator  problem  solving.  This,  in  turn,  may 
enable  development  of  more  sophisticated  and  effective  tactics.  Certainly,  current 
tasks  of  target  detection,  localization,  and  classification  would  improve.  Reductions  m 
operator  training  and  increases  in  retention  could  also  be  significant,  and  the 
effectiveness  of  each  sonar  operator  could  be  multiplied  as  well. 

This  technology  could  also  be  applied  effectively  to  ASW  tactical  support  by 
providing  the  shipboard  ASW  officer  (or  submarine  approach  officer.  Figure  7)  with  a 
3-D  image  of  the  ocean  environment,  his  weapons  engagement  envelops,  sensor 
coverage  volumes,  and  the  hostile  submarines  that  have  been  identified. 

3-D  display  technology  could  also  provide  E-2C  controllers  with  3-D  aircraft 
engagement  information  including  launch  envelopes,  detection  system  coverage,  etc. 
(Figure  8), 

It  could  be  applied  to  the  display  of  battle  management  information  (Figure  9) 
where  location  and  status  of  assets  determines  weapons  and  sensor  coverage. 

A  simpler  application  would  be  for  air  traffic  control  afloat  or  ashore  (Figure  10) 
where  only  limited  organic  sensor  information  must  be  presented.  Such  a  system 
might  also  benefit  from  the  incorporation  of  real-time  voice  control  of  the 
communications  system. 


The  Jgchngtpgies 

Four  technologies  are  maturing  sufficiently  to  permit  a  serious  investigation  of 
their  operational  use  in  an  integrated  information  display  system.  They  are  high¬ 
speed  graphics  computers,  miniature  high-resolution  displays,  3-D  position  sensors, 
and  digital  3-D  audio. 

COMPUTERS 

High-speed  graphics  computers  are  capable  of  generating  acceptably  realistic 
imagery  that  is  updated  fast  enough  to  present  an  operator  with  useful  information  in 
real  time.  In  the  past,  such  capabilities  were  limited  to  massive  and  costly 
supercomputers.  Fortunately  the  highly  competitive  nature  of  this  industry  continues  to 
push  cost  down  and  performance  up.  Present  high-speed  graphics  computers 
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surpass  the  performance  of  these  older  machines  at  a  fraction  of  the  cost  and  size.  A 
few  years  ago  a  small  computer  that  could  run  at  5  million  floating  point  operations  per 
second  (MFLOP)  and  generate  about  6,000  polygons  per  second  was  the  hottest  thing 
on  the  market.  Today  that  would  be  a  “low-end"  machine.  The  “high-end"  graphics 
computers  today  run  about  30  times  faster  (180,000  polygons  per  second)  at  a 
comparable  price.  All  of  these  machines  are  using  multiple  processors  to  achieve 
these  speed  and  cost  improvements  with  large  supercomputers  now  operating  with 
thousands  of  processors.  The  next  ten  years  will  see  a  continued  increase  in 
performance  with  a  corresponding  decrease  in  cost  and  size.  Today's  high-speed 
graphics  computers  will  seem  slow  by  the  time  a  3-D  display  system  is  ready  to  enter 
the  Fleet. 

DISPLAYS 

Miniature  high-resolution  displays  evolved  from  weapons  control  and  sensor 
display  requirements  for  military  aircraft.  Much  of  this  work  was  pioneered  at  the  Naval 
Weapons  Center  at  China  Lake  under  the  AGILE  Missile  programmed  by  the  Air  Force 
at  Wright-Patterson  AFB.  Although  the  Navy  fielded  the  first  operational  helmet- 
mounted  display  (HMD),  the  most  widely  used  of  these  systems  can  be  found  aboard 
combat  helicopters.  These  systems  now  use  cathode  ray  tubes  (CRTs)  as  small  as  0.5 
inches  in  diameter  with  1.000  line  resolution.  Approximately  875  to  1000  line 
resolution  is  required  for  “seamless”  imagery  in  an  HMD  with  an  average  field  of  view. 
By  way  of  comparison,  a  photograph  Is  approximately  4,000  pixels  high.  All  of  these 
small  CRTs  are  either  monochrome  or  black  and  white.  There  appears  to  be  a 
technology  barrier  against  making  full-color  CRTs  of  this  size,  although  there  has  not 
been  a  strong  military  or  commercial  requirement  to  push  small  color-CRT  technology. 

Small  monochrome  CRTs  from  portable  TVs  are  in  use  in  boom-mounted 
stereoscopic  displays.  Unfortunately,  the  process  of  magnifying  the  imagery  from 
these  CRTs  for  a  stereoscopic  display  with  an  average  field  of  view  causes  resolution 
problems.  The  use  of  standard  TV  technology  (550  line  NTSC  video)  results  in  very 
noticeable  black  stripes  across  the  image  due  to  the  return  stroke  blanking  used  in 
conventional  raster  scan  video.  The  use  of  high  definition  TV  would  reduce  or 
eliminate  this  problem,  but  the  day  that  such  displays  are  available  in  the  small- 
portable  market  is  a  long  way  off. 

Color  active-matrix  Liquid  Crystal  Displays  (LCDs)  for  the  miniature  TV  market 
are  now  being  used  for  displays  in  the  majority  of  the  commercial  stereoscopic 
systems.  These  displays  have  recently  doubled  in  linear  resolution  to  the  point  of 
being  comparable  with  standard  television  resolution  (LCD  image  height  of  480  pixels 
vs.  500  lines  of  raster-scan  video  for  conventional  TV).  Although  this  is  a  great 
improvement  over  the  previous  generation  of  240  pixel-high  LCDs,  this  resolution  is 
still  low  enough  that  the  image  appears  very  grainy  when  magnified  as  part  of  a 
stereoscopic  display  system.  Unfortunately,  higher  resolution  LCDs  are  not  likely  to 
emerge  from  the  commercial  market  unless  the  demand  increases  by  several  orders  of 
magnitude.  The  technology  for  creating  a  1.0  inch  diagonal  1 ,024  x  1 ,024  pixel  active- 
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-natrix  color  LCD  is  available,  but  the  fabrication  costs  are  estimated  to  be  about 
.lillion  dollars  for  the  process  development  and  production  set  up. 

3-D  POSITION  SENSORS 

The  most  prevalent  3-D  position  sensor  system  in  use  today  is  the  Polhemus 
tracker.  This  device  uses  a  magnetic  field  generator  affixed  to  the  local  environment 
and  a  field  sensor  attached  to  the  device  to  be  monitored.  A  computer  system 
monitors  the  field  sensor  and  calculates  the  position  and  attitude  of  the  sensor  based 
on  the  nature  of  the  portion  of  the  magnetic  field  in  which  it  is  immersed.  These  units 
are  small  and  rugged,  but  they  are  sensitive  to  the  presence  of  ferrous  metals  which 
can  distort  the  magnetic  field  geometry.  Work  is  being  done  to  reduce  this  problem 
and  to  speed  up  the  operation  of  the  system.  Other  position  trackers  are  now  reaching 
the  commercial  market  however.  One  unit  uses  multiple  ultrasonic  sensors  and 
sources  to  monitor  position  and  attitude.  An  analogous  multi-element  infrared  system 
.s  also  available.  These  systems  are  not  sensitive  to  electromagnetic  interference,  but 
they  do  have  their  own  limitations.  The  Polhemus  system  is  the  only  one  that  is 
currently  incorporated  into  3-D  control  devices. 

3-D  AUDIO 

Of  all  the  technologies  that  are  supporting  the  ability  to  effectively  display 
information  in  3-D,  this  is  perhaps  the  newest.  It  is  an  outgrowth  of  research  into 
human  auditory  perception.®-  T  ®  Initial  research  demonstrated  the  importance  of  the 
shape  of  the  outer  ear  to  the  localization  of  sounds.  Subsequently,  the  ability  to  modify 
sounds  using  experimentally  derived  transform  functions  was  achieved.  The  transform 
functions  were  derived  by  measuring  the  distortion  of  known  broad-band  sounds  due 
to  outer  ear  shape.  An  undistorted  sound,  after  processing  by  a  transform  function 
incorporated  into  circuitry,  would  seem  to  have  emanated  from  any  desired  location. 
This  made  it  possible  to  alter  a  sound  so  that  it  would  seem  to  have  come  from  any 
chosen  location  in  space  external  to  the  listener.  Recent  work  at  NASA,  Ames  resulted 
in  these  transforms  being  incorporated  into  digital  circuitry  and  married  to  computers 
that  could  simultaneously  monitor  the  orientation  of  the  operator’s  head  (using  a 
Polhemus  tracker)  and  transform  a  sound  so  that  it  would  seem  to  be  stationary  while 
the  operator  moved.  NOSC  has  the  first  advanced  digital  system  of  this  type  and  is 
conducting  research  into  its  application  to  various  operational  tasks. 

Conclusions 

These  technologies  are  already  being  applied  to  numerous  civilian  tasks 
including  architecture,  mechanical  design  in  the  automotive  and  aerospace  industries, 
pharmaceuticals  research,  medical  imagery  for  3-D  display  of  CAT  scan,  NMR,  and 
ultrasonic  medical  data,  education,  and  functional  aids  for  the  handicapped. 
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If  these  new  technologies  are  applied  to  the  tactical  Naval  environment  tneir 
impact  on  operational  effectiveness  could  be  dramatic  and  their  impact  on  framing 
methods  and  systems  may  be  equally  dramatic. 

History  has  taught  us  that  the  requirement  to  operate  a  global  Navy  will  not 
diminish  at  a  rate  commensurate  with  the  decrease  in  our  assets.  Budgets  and 
manpower  levels  will  probably  continue  to  decrease  over  the  next  decade  or  two. 
Therefore  the  workload  for  each  person  will  increase,  and  the  tactical  cost  of  loosing 
one  platform  out  of  a  reduced  inventory  will  be  magnified.  Systems  that  can  improve 
operational  performance  while  requiring  lower  manning  will  be  critical  and  those 
systems  that  can  achieve  the  greatest  performance  improvements  at  the  lowest  cost 
will  be  essential. 
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Multidimensional  audio  information  is 
presented  as  abstract  visual  data  in  2-D 
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